Social networking services are accessed by users to communicate with each other, share their interests, upload images and videos, create new relationships, etc. Social networking services typically operate in a distributed computing environment with data being distributed among one or more server clusters that are each located in one of multiple data centers. A server cluster is a grouping of server computing devices (“servers”). When a user of a social networking service sends a query to request and/or process data, a front-end load balancing server can be the first server to receive the user request. The front-end load balancing server usually routes the request to a front-end server (e.g., web server) in one of the server clusters. The front-end server can identify one or more back-end application services to help respond to the user request. The front-end server generates one or more service requests to the back-end application services based on a routing table published by a back-end routing service.
A conventional back-end routing service faces different challenges than the front-end load balancing server. For example, polling performance metrics from back-end servers are much more resource intensive compared to polling the performance metrics of front-end servers. The conventional back-end routing service publishes, periodically, a routing table based on known front-end service request statistics and back-end server performance statistics. The conventional back-end routing service generally reduces amount of traffic to a back-end server whenever that back-end server is over its capacity limit. This conventional technique causes the back-end servers to frequently experience oscillations, where for example, an available back-end server often times receives a sudden overload of service requests followed by a drought of application service requests leading to sub-optimal usage.
The figures depict various embodiments of this disclosure for purposes of illustration only. One skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles of embodiments described herein.